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ABSTRACT 

Traditional stereo techniques determine range by relating two images of an object viewed 
from different directions. If the correspondence between picture elements is known, then distance to 
the object can be calculated by triangulation. Unfortunately, it is difficult to determine this 
correspondence. 

This paper introduces a novel technique called photometric stereo. The idea of 
photometric stereo is to vary the direction of the incident illumination between successive views 
while holding the viewing direction constant. This provides enough information to determine 
surface orientation at each picture element. Since the imaging geometry does not change, the 
correspondence between picture elements is known a priori. This stereo technique is photometric 
because it uses the intensity values recorded in a single picture element, in successive views, rather 
than the relative positions of features. ' 
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1. Introduction 

Work on image understanding has led to a need to model the imaging process. One 
aspect of this concerns the geometry of image projection. Less well understood is the radiometry of 
image formation. Relating the intensity values recorded in an image to object relief requires a 
model of the way surfaces reflect light. 

A reflectance map is a convenient way to incorporate a fixed scene illumination, object 
photometry and imaging geometry into a single model that allows image intensity to be related 
directly to surface orientation. This relationship is not functional since surface orientation has two 
degrees of freedom and image intensity provides only one measurement. Local surface topography 
cannot, in general, be determined by the intensity value recorded at a single image point. In order 
to determine object shape, additional information must be provided. 
y% This observation has led to a novel technique called photometric stereo in which surface 

orientation is determined from two or more images. Traditional stereo techniques determine range 
by relating two images of an object viewed from different directions. If the correspondence 
between picture elements is known, then distance to the object can be calculated by triangulation. 
Unfortunately, it is difficult to determine this correspondence. The idea of photometric stereo is to 
vary the direction of the incident illumination between successive views while holding the viewing 
direction constant. This provides enough information to determine surface orientation at each 
picture element. Since the imaging geometry does not change, the correspondence between picture 
elements is known a priori. This stereo technique is photometric because it uses the intensity values 
recorded at a single picture element, in successive views, rather than the relative positions of 
features. 



2. The Reflectance Map 

The fraction of light reflected by a surface in a given direction depends upon the optical 

properties of the object material, the surface microstructure and the spatial and spectral distribution 

and state of polarization of the incident light. A key photometric observation is: 

Mo matter how complex the distribution of incident illumination, for most surfaces, the 
fraction of the incident light reflected in a particular direction depends only on the 

surface orientation. 

The reflectance characteristics of an object material can be represented as a function 
<£(i,e,g) of the three angles i, e and g defined in figure I. These are called, respectively, the 
incident, emergent and phase angles. The angles i, e and g are defined relative to the object 
surface. . <Mi,e,g) determines the ratio of radiance to irradiance measured per unit surface area, 
per unit solid angle, in the direction of the viewer. The reflectance function defined here is related 
y»> to the bi-directional reflectance-distribution function defined by the National Bureau of Standards 

[Nicodemus et al 77]. 

Image forming systems perform a perspective transformation as illustrated in figure 2(a). 
If the size of the objects in view is small compared to the viewing distance, then the perspective 
projection can be approximated as an orthographic projection as illustrated in figure 2(b). 
Consider an image forming system that performs an orthographic projection. To standardize the 
imaging geometry, it is convenient to align the viewing direction with the negative z-axis. Also, 
assume appropriate scaling of the image plane so that object point (x,y,z) maps onto image point 
(u,v) with u = x and v = y. With these assumptions, image coordinates (x,y) and object 
coordinates (x,y) can be referred to interchangeably. 

If the equation of a surface is given explicitly as: 
z = f(x,y) 
then a surface normal is given by the vector: 



[3f(x,y)/6x,af(x,y)/ay,-l]. 

If parameters p and q are defined by: 

p = 6f(x,y)/3x and q = 6f(x,y)/8y 
then the surface normal can be written as [p,q,-l]. The quantity (p,q) is called the gradient, and 
gradient space is the two-dimensional space of all such points (p,q). Gradient space is a 
convenient way to represent surface orientation. It has been used in scene analysis [Mackworth 73]. 
In image analysis, it is used to relate the geometry of image projection to the radiometry of image 
formation [Horn 77], , 

One important simplification inherent in the assumption of an orthographic projection is 
that the viewing direction, and hence the phase angle g, is constant for all object points. Thus, for 
a standard light source and viewer geometry, the ratio of radiance to irradiance depends only on 
/*>. gradient coordinates p and q. 

Further, suppose each surface element receives the same incident illumination. Then, the 
amount of the incident light reflected in a particular direction depends only on the surface 
orientation. The assumption of an orthographic projection also allows one to relate the amount of 
light reflected per unit surface area per unit solid angle in the direction of the viewer directly to 
image intensity. Thus, for the given imaging geometry, for a given distribution of incident 
illumination and a given object material, the image intensity corresponding to a surface point with 
gradient (p,q) is unique. 

The reflectance map R(p,q) determines image- intensity as a function of p and q. A 
reflectance map captures the surface photometry of an object material for a particular light source, 
object surface and viewer geometry. It explicitly incorporates both the geometry and radiometry of 
image formation into a single model. 



/""*\ 



/"■v 



If the viewing direction and the direction of incident illumination are known, then 
expressions for cos(i), cos(e) and cos(g) can be derived in terms of gradient space coordinates 
p and q. Suppose vector [p s ,q s ,-l] defines the direction to the light source. Then: 



#■% l + pp s + qq s 

cos(i) ^ Si 



Vl •+ P 5 Z + q s S/l 



5 ■ ' «s v 1 + P + <r 
cos(e) = 



yjl - - 2 - - z 



+ P + q 
cos(g) = 



V 1 + P s 2 + q s* 

These expressions allow one to transform an arbitrary surface photometric function <£(i,e,g) into 
a reflectance map function R(p,q). 
f*\ Reflectance maps can be determined empirically, derived from phenomenological models 

of surface reflectivity or derived from analytic models of surface microstructure. One simple, 
idealized model of surface reflectance is given by: 

<£ a (i,e,g) = p cos(i) 
This reflectance function corresponds to the phenomenological model of a perfectly diffuse 
(lambertian) surface which appears equally bright from all viewing directions. Here, p is a 
reflectance factor and the cosine of the incident angle accounts for the foreshortening of the surface 

as seen from the source. The corresponding reflectance map is given by: 

p (1 + pp. + qq.) 



R a (p,q) 



s H ^s' 



Vl + p. 2 + q 2 Vl + P 2 + q 2 



's M s 
A second reflectance function,, similar to that of materials in the maria of the moon and 

rocky planets, is given by: 

<£ b (i.e,g) = p cos(i)/cos(e) 

This reflectance function corresponds to a surface which reflects equal amounts of light in all 
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directions. The cosine of the emergent angle accounts for the foreshortening of the surface as seen 
from the viewer. The corresponding reflectance map is given by: 



Reflectance maps are independent of the shape of the objects being viewed. To 
emphasize that a reflectance map is not an image, it is convenient to present R(p,q) as a series of 
"iso-brightness" contours in gradient space. Figure 3 and figure 4 illustrate the two simple 
reflectance maps R a (p,q) and R b (p,q), defined above, for the case p s = 0.7 q s = 0.3 and 
p = 1. 

3. Reflectance Map Techniques 
-^ Using the reflectance map, the basic equation describing the image-forming process can 

be written as: 

Kx.y) = R(p,q) (1) 

This equation has been used in image analysis to explore the relationship between image intensity 
and object relief. Direct solution is tedious. One can think of (1) as one equation in the two 
unknowns p and q. Determining object shape from image intensity is difficult because (1) is 
underdetermined. In order to calculate shape, additional assumptions must be invoked. 

Recent work has helped to make these assumptions explicit. For certain materials, such as 
the material of the maria of the moon, special properties of the reflectance map simplify the 
solution [Rindfleisch 66] [Horn 75] [Horn 77], Other methods for determining object shape from 
image intensity embody assumptions about surface curvature [Horn 77] [Woodham 77]. Simple 
surfaces have been proposed for use in computer aided design [Huffman 75]. When properties of 
surface curvature are known a priori, these can be exploited in image analysis [Woodham 78], This 
is useful, for example, in industrial inspection since there are often constraints on surface curvature 



imposed by the drafting techniques available for part design and by the fabrication processes 
available for part manufacture. One purpose of these studies is to deepen our understanding of 
what can and can not be computed directly from image intensity. 

Other reflectance map techniques use (1) directly to generate shaded images of surfaces. 
This has obvious utility in graphic applications including hill-shading for automated cartography 
[Horn 76] and video input for a flight simulator [Strat 78], Synthesized imagery can be registered 
to real imagery to align images with surface models. This technique has been used to achieve 
precise alignment of LANDSAT imagery with digital terrain models [Horn & Bachman 77]. 

4. Photometric Stereo 

Photometric stereo is a novel reflectance map technique that uses two or more images to 
^•^ solve (1) directly. The idea of photometric stereo is to vary the direction of incident illumination 

between successive views while holding the viewing direction constant. Suppose two images 
Ij(x,y) and I 2 (x,y) are obtained by varying the direction of incident illumination. Since there 
has been no change in the imaging geometry, each picture element (x,y) in the two images 
corresponds to the same object point and hence to the same gradient (p,q). The effect of varying 
the direction of incident illumination is to change the reflectance map R(p,q) that characterizes the 
imaging situation. ' 

Let the reflectance maps corresponding to ^(x.y) and I 2 (x,y) be Rj(p,q) and 
RgCP.q) respectively. The two views are characterized by two independent equations: 

I^x.y) = R^p.q) (Z) 

I 2 (x,y) = R z (p,q) (3) 

Two reflectance maps R^p.q) and R z (p,q) are required. But, if the phase angle g is the same in 
both views (i.e., the illumination is simply rotated about the viewing direction), then the two 
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reflectance maps are rotations of each other. 

For reflectance characterized by R b (p,q) above, (2) and (3) are linear equations in p 
and q. If the reflectance factor p is known, two views are sufficient to uniquely determine surface 
orientation at each image point, provided the directions of incident illumination are not collinear. 
Here, (2) and (3) suggest that a 90° angle between the directions of incident illumination would be 
optimal for photometric stereo. 
• ' In general, equations (2) and (3) are nonlinear so that more than one solution is 

possible. One idea would be to obtain a third image: 

I 3 (x,y) = R 3 (p,q) (4) 

to overdetei mine the solution. 

For reflectance characterized by R g (p,q) above, three views are sufficient to uniquely 
f\ determine both the surface orientation and the reflectance factor p at each image point [Horn 78]. 

Let I = [Ij, I 2 , I 3 ]' be the column vector of intensity values recorded at a point (x,y) in each of 
the three views ( ' denotes vector transpose). Further, let 

11 1 = [n ll' n 12' n 13^ 

"2 = tn 21' n 22' n 23 ] ' 
n 3 = [n 31 ,n3 2 ,n33]' 

be unit column vectors which point in the direction of the three positions of the incident 

illumination. Construct the matrix N where: 



N 



n ll n 12 "13 
n 21 n 22 n 23 
n 31 n 32 "33 



Let n = [n 1 ,n 2 ,n 3 ] 1 be the column vector corresponding to a unit surface normal at (x,y). 
Then, 

f*y I = p N n 

so that, p n = N" 1 I 



provided the inverse N exists. This inverse exists if and only if the three vectors n,, n? and n, 
do not lie in a plane. In this case, the reflectance factor and unit surface normal at (x,y) are given 
by: 

P - IN -1 I| and 

n = (l/p)N _1 I (5) 

Unfortunately, since the sun's path across the sky is planar, this simple solution does not apply to 
outdoor images taken at different times during the same day. 

Equation (5) suggests that three mutually orthogonal directions of incident illumination 
would be optimal for lambertian reflectance. In any stereo technique, however, there is some 
trade-off to acknowledge. In photometric stereo, choosing a larger phase angle g leads to more 
accurate solutions. At the same time, a larger phase angle causes a larger portion of gradient space 
/"\ t°. lie in the shadow region of one or more of the sources. A practical compromise is achieved by 

using four light sources and a relatively large phase angle [Woodham 78]. Solutions are accurate 
and most of gradient space lies in regions illuminated by at least three of the sources. Three image 
intensity measurements overdetermine the set of equations and establish a unique solution. 

The images required for photometric stereo can be obtained by explicitly moving a single 
light source, by using multiple light sources calibrated with respect to each other or by rotating the 
object surface and imaging hardware together to simulate the effect of moving a single light source. 
The equivalent of photometric stereo can also be achieved in a single view by using multiple 
illuminations which can be separated by color. 

Photometric stereo is fast. It has been developed as a practical scheme for environments 
in which the nature and position of the incident illumination is known or can be controlled. Initial 
computation is required to determine the reflectance map for a particular experimental situation. 
Once calibrated, however, photometric stereo can be reduced to simple table lookup and/or search 
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operations. 



5. Applications of Photometric Stereo 

Photometric stereo can be used in two ways. First, photometric stereo is a general 
technique for determining surface orientation at each image point. For a given image point (x,y), 
the equations characterizing each image can be combined to determine the corresponding gradient 
(p,q) 

Second, photometric stereo is a general technique for determining object points that have, 

a particular surface orientation. This use of photometric stereo corresponds to interpreting the basic 

image-forming equation (1) as one equation in the unknowns x and y. For a given gradient (p,q), 

the equations characterizing each image can be combined to determine corresponding object points 

.f*\ U,y). 

This latter use of photometric stereo is appropriate for the so called industrial 
bin-of-parts problem. The location in an image of "key" object points is often sufficient to 
determine the position and orientation of a known object tossed onto a table or conveyor belt. 

A particularly useful special case concerns object points whose surface normal directly 
faces the viewer (i.e. object points with p = and q = 0). Such points form a unique class of 
image points whose intensity value is invariant under rotation of the incident illumination about the 
viewing direction. Object points with surface normal directly facing the viewer can; be located 
without explicitly determining the reflectance map R(p,q). Whatever the nature of the function 
R(p,q), the value of R(0,0) is not changed by varying the direction of incident illumination, 
provided only that the phase angle g is held constant. 



These applications of photometric stereo are illustrated using a simple, synthesized 
example. Consider a sphere of radius r centered at the object space origin. The explicit 
representation of this object surface, corresponding to the viewing geometry of figure 2(b), is given 
by: 

z = f(x,y) = -\r 2 - x Z - y Z (6) 

The gradient coordinates p and q are determined by differentiating (6) with respect to x and y. 
One finds: 

p = -x/z and q = -y/z 

Suppose that the sphere is made of a perfectly diffusing object material and is 

illuminated by a single distant point source at gradient point (P s ,q s ). Then, the reflectance map is 

given by R a (p,q) above so that the corresponding image is: 

f"\ ( if x Z + y 2 > r Z 

r < x -y) = < max{0,R a (-x/z,-y/z)} otherwise < 7 > 

Equation (7) generates image intensities in the range to p. In the example below, r = 60 and 

P = 1. 

Multiple images are obtained by varying the position of the light source. Consider three 
different positions. Let the first be p s = 0.7 and q g = 0.3, as above. Let the second and third 
correspond to rotations of the light source about the viewing direction of -1Z0° and +120° 
respectively (i.e., p s = -0.610, q s = 0.456 and p s = -0.090, q s = -0.756). Let the three 
reflectance maps be Rj(p,q), R 2 (p,q) and R 3 (p,q). The phase angle g is constant in each case. 
Let the corresponding images generated by (6) be I^x.y), I 2 (x,y) and I 3 (x,y). 

First, consider image point x = 15, y = 20. Here, I^x.y) = 0.942, I 2 (x,y) = 0.723 
and I 3 (x,y) = 0.505. Figure 5 illustrates the reflectance map contours R.(p,q) = 0.942, 
O R 2 (P,q) = 0.723 and R 3 (p,q) = 0.505. The point p = 0.275, q = 0.367 at which these three 

contours intersect determines the gradient corresponding to image point x = 15, y =' 20. 
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Second, consider gradient point p = 0.5, q * 0.5. Here, R^p.q) = 0.974, 
RgtP.q) = °- 6 00 and R 3 (p.q) = 0.375. Figure 6 illustrates the image intensity contours 
I^x.y) = 0.974, I z (x,y) = 0.600 and I 3 (x,y) = 0.375. The point x = 24.5, y = 24.5 at 
which these three contours intersect determines an object point whose gradient is p = 0.5, q = 0.5. 

Finally, Figure 7 repeats the example given in Figure 6 but for the case p = 0, q = 0. 
Here, R 1 (p,q) = R 2 (p,q) = R 3 (p,q) = 0.796. Object points with surface normal directly facing 
the viewer form a unique class of points whose image intensity is invariant for rotations of the light 
source about the viewing direction. The point x = 0, y = at which these three contours intersect 
determines an object point with surface normal directly facing the viewer. This result holds even if 
the form of R(p,q) is unknown. 

6. Conclusions 

Surface orientation at a point on an object can be determined from the image intensities 
obtained under a fixed imaging geometry but with varying lighting conditions. Because the images 
used in photometric stereo arise from the same viewpoint, there is no difficulty determining points 
in one image which correspond to the same object point in another image. The calculations 
required in photometric stereo are local and can be implemented in a straightforward manner. 
Photometric stereo does not depend on the recognition of features in a scene but instead makes use 
of the intensity values recorded at each picture element. 
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Figure 1 Defining the three photometric angles i, e and g. The incident angle i is the an*le 
between the incident ray and the surface normal. The view angle e is the angle between the 
emergent ray and the surface normal. The phase angle g is the angle between the incident and 
emergent rays. 
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Perspective Projection 
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Orthographic Projection 
u = x v = y 



(x,y,z) 
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Figure 2 Characterizing image projections. Figure 2(a) illustrates the well-known perspective 
projection [Note: to avoid image inversion, it is convenient to assume that the image plane lies in 
front of the lens rather than behind it] For objects that are small relative to the viewing distance 
the image projection can be modeled as the orthographic projection illustrated in figure 2(b) In an 
orthographic projection, the focal length f is infinite so that all rays from object to image are 
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Figure 3 The reflectance map R a (p,q) for"7Lambertian surface illuminated from gradi 
P s = 7 and q s = 0.3 (with p ■ 1.0), The reflectance map is plotted as a series of 
(spaced 0.1 units apart). 



ient point 
contours 
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Figure 4 The reflectance map R b (p, q ) for a perfect 6^^^^^™ relent 
pent p = 0.7 and q$ . 0.3 (with p . l.o). The reflectance map „ pitted as a s! es o 
contours (spaced 02 units apart). P P S a Senes of 
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R2(p,q) = 0.723 



Rl(p,q) = 0.942 
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Figure 5 Determining the surface orientation (p,q) at a given image point (x,y) Three 
(superimposed) reflectance map contours ate intersected where each contour corresponds to the 
intensity value at (x,y) obtained from three separate images (taken under the same imaging 
geometry but with different light source position). I,(x,y) = 0.942, I 7 (x,y) = 0.723 and 
I 3 (x,y) = 0.505. 
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Iz(x,y) 



0.974 
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I3(x,y) = 0.375 
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Figure 6 Determining image points (x,y) whose surface orientationls a glv^rTgratonpTqr 
Three (superimposed) image intensity contours are intersected where each contour corresponds to the 
value at (p,q) obtained from three separate reflectance maps. (Each reflectance map characterizes 

the same imaging geometry but corresponds to a different light source position.) R,(p,q) = 0.974 
R 2 (p,q) = 0.600 and R 3 (p,q) = 0.375. 
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Iz(x,y) = 0.796 



Ii(x,y) = 0.796 




Figure 7 T)etermining image points IvTbsTlu^^^ the viewer. Three 

(superimposed) image intensity contours are intersected where each contour corresponds to the value 
at (0,0) obtained from three separate reflectance maps. (Each reflectance map characterizes the 
same imaging geometry but corresponds to a different light source position.) Note that the 
reflectance map value at (0,0) does not change with light source position (provided the phase 
angle g is held constant). 



